Motivation

A challenge:

• Quality of service provision is low in many developing countries

• Recruiting and incentivizing service providers is difficult

A case:

• Agricultural extension workers
• The 2013–2016 Ebola virus epidemic in West Africa
• Caught many by surprise, no early warning. Need for stronger decentralized bureaucracy.

• \(>\) 7/10 emerging infectious diseases spill over from animals

Motivation

Promising work on improving service provision:

1. Recruitment

2. Community monitoring

3. Financial incentives

We show:

• evidence that both 2. and 3. matter with limited crowding out.
• no good news on 1.
• we use a structural model to explore why and when these might be more effective.

Past Work on Recruitment

Theoretically:

• Intrinsic motivation reduces shirking (Francois 2000)

Empirically:

• Higher wage offers attracts more qualified and competent candidates (Dal Bó, Finan, and Rossi 2013)

• Big five personality traits reduce shirking (Callen et al 2015)

• Pro-sociality correlates to health worker performance (Deserranno 2017)

• Do career benefits attract talent at the expense of prosocial motivation? (Ashraf, Bandiera, and Lee 2020)

Past Work on Community Monitoring

• View that bottom-up pressure by empowered citizens is key to improved service delivery

• Overall mixed effects:

  • In health: Björkman and Svensson (2009); Björkman and Svensson (2017); Raffler et al. (2021); Christensen et al (2021)

  • In education: Banerjee et al. (2010); Barr et al. (2012); Pradhan et al (2014); Andrabi et al. (2017)

Past Work on Financial Incentives

By and large a success:

• Education: Muralidharan and Sundararaman (2011), Kahn et al (2014), Glewwe et al (2010), De Ree et al (2010), Leaver et al (2021)

• Health status: Miller et al (2012) Mohanan et al. (2021)

• Service delivery: Gertler and Vermeersch (2012), Desserano et al (2022)

• Service provider inputs: teachers (Duflo et al, 2012, Leaver et al 2021), nurse attendance [banerjee2006addressing]

The policy questions

1. Are community monitoring mechanisms more or less effective than performance-based payment schemes?

2. Are there interaction effects between recruitment strategies and incentive mechanisms?

• E.g. community monitoring may be less effective when service providers are selected by the state

Design

We study these questions in the context of a randomized agricultural intervention in Sierra Leone.

Core design elements (more on context to come)

Model

A worker has type \(\theta = (\eta,\mu)\) where:

• \(\eta \in \{ \underline{\eta} , 1 \}\) denotes embeddedness
  
• \(\mu \in \{ 0 ,\overline{\mu} \}\) denotes intrinsic motivation

Drawn according to:

So \(\rho\) governs the correlation between embeddedness and motivation: embedded people are possibly more effective, but are they more likely to be highly motivated?

T1 and type

• Under chief selection (\(T_1=0\)) the worker is a random draw from the candidate pool.
• Under villager selection (\(T_1=1\)), the worker is drawn from the pool with high embeddedness.

So paramount chief is more likely to select workers with high intrinsic motivation if and only if \(\rho < 0\).

Performance

• Workers choose effort \(e\)

• Performance depends on effort and embeddedness: \[\pi = e \eta\]

• Stochastic performance signal \(\sigma\), which is high (\(\sigma=1\)) with probability \(\pi\) or low (\(\sigma=0\)) with probability \(1-\pi\)

• Cost of effort: \(c(e) = \frac{e^2}{2\kappa}\)

Villager (Monitoring)

• A representative villager selects a social sanctioning plan, which depend on the performance signal \(\sigma\): \[s=(s(0),s(1)) \in [-T_2, T_2]^2.\]

Interpretable as commitments to reward (\(s(\pi)<0\)) or punish (\(s(\pi)>0\)) performance.

Punishment is possibly psychologically costly, according to \(\gamma\).

\[u(\pi, s) = \pi - \gamma s .\]

• \(\zeta\) is probability punishment is not costly for the villager

CAHW utility

Under pay-for-performance (\(T_3=1\)) and high performance signal (\(\sigma=1\)), the worker receives a payment normalized to 1

In addition:

• \(\phi\): sensitivity to social sanctions
• \(\mu\): intrinsic benefits from performing well

Putting it all together:

\[w(e;s,\sigma;\theta) = \sigma ( T_3 + \mu ) - \phi s(\sigma) T_2 - \frac{e^2}{2 \kappa},\]

Timeline.

The sequence of events is as follows:

1. worker is selected by either village (\(T_1=1\)) or chief (\(T_1=0\)).
2. villagers and worker learn whether the latter is subject to community monitoring (\(T_2=1\)) and/or P4P (\(T_3=1\)).
3. if \(T_2=1\), villager with guilt parameter \(\gamma\) commits to a conditional plan of social sanctioning \(\{s(\sigma)\}_{\sigma \in \{0,1\}}\).
4. The worker chooses effort \(e\).
5. The worker’s performance signal \(\sigma\) is realized.
6. Payoffs are realized and the game ends.

Solution concept

An equilibrium is a pair of strategies \(\{ s,e(s;\theta) \}\) such that

1. effort is individually rational for each type given the sanctioning strategy
2. the sanctioning strategy maximizes the villager’s expected payoff given anticipated effect on the workers’ effort \(\hat{e}(s;\theta)\), and
3. expectations are correct (\(\hat{e}(s;\theta)=e(s;\theta)\) for each \(\theta\)) :

\[e(s;\theta) \in \arg \max_{e \in [0,1]} \mathbb{E}_{\pi } \Big\{ w(e,s,\sigma;\theta) \Big\}\] \[s \in \arg \max_{\{ (s(0),s(1)) \in [-T_2,T_2]^2 \}} \mathbb{E}_{\theta,\pi} \Big\{ u(\pi,s) \mid \hat{e}(s;\theta) \Big\}\]

Results

• Sanctioning.
  • following a high signal: always praise the worker
  • following a low signal: choose maximal shaming \(s(0)=1\) if susceptibility to guilt is low (\(\gamma=0\)) and minimal shaming \(s(0)=-1\) otherwise.
• Effort:

\[e(s^*;\gamma,\theta) = \begin{array} \kappa \eta (\mu + T_3 + T_2 2 \phi) & \mbox{if} \quad \gamma =0 \\ \kappa \eta (\mu + T_3) \quad & \mbox{otherwise}. \end{array}\]

Results

Full closed form solution for all treatment conditions:

Hypotheses inspired by model

Hypothesis 1: Village selection increases performance.

• Explanation: Requires gains from embeddedness not crowded-out by any reductions in intrinsic motivation

Hypothesis 2: Community monitoring increases performance.

• Explanation: Community monitoring (T2=1) incentivizes worker performance through the associated probability of being publicly shamed for poor performance.

Hypothesis 3: Pay-for-performance increases performance.

• Explanation: Pay for performance (T3=1) increases performance due to the probability of receiving a monetary reward.

Hypotheses

Hypothesis 4: There is a positive interaction between village selection and pay-for-performance.

• Explanation: More productive workers are more responsive to extrinsic incentives (T3=1).

Hypothesis 5: There is a positive interaction between village selection and community monitoring.

• Explanation: More productive workers (i.e. those with higher embeddedness, T1=1) as it is easier for villagers to commit to sanctioning (as long as embeddedness does not crowd-out intrinsic motivation too much).

• [Hypothesis 6: There is a positive interaction between community monitoring and pay-for-performance.

Explanation: The expected improvement in performance driven by the monetary incentive (T3=1) makes it easier for the village to commit to punish low performance (T2=1).]

Context: Sierra Leone

Poor service provision

• In 2020, U5 mortality is 108/1,000 live births (45/48)

Threat of zoonosis is high and expected to increase

• Animals are essential for livelihoods (90% keep animals)

• Low information and bad animal husbandry practices

• Animal diseases are highly contagious

• Impacts on households are enormous and largely preventable

Context

• In the wake of the Ebola epidemic, together with the GoSL, we implement a “One Health” intervention, addressing animal and human health

• 360 villages across 7 chiefdoms in Kono district

Context

• Within the One Health program: we randomize recruitment and incentives across villages

One Health Program

• Co-designed with GoSL’s MOH and MAF

• Implemented from July 2017 to July 2019

• Recruitment of Community Animal Health Worker: CAHW

• 21-day training, implemented by veterinarians

• Tool kit and drugs for treatment of basic animal diseases

• Creation One Health platform

• CAHW:

  • provides basic animal health services
  • demonstrates and provides information on best practices
  • reports suspected disease symptoms to MAF supervisors

One Health Program: 21-day training

One Health Program: Animal disease reporting form

One Health Program: Results

• March and April 2020: Household endline survey (344/363 villages)

• Large increases in animal stocks (22%)

One Health Program

• Strong enthusiasm for the program

  • reporting throughout the period and beyond

• No impact on human health, but impacts on intermediary outcomes

• From planner’s perspective it matters if we can improve CAHW performance:

  1. Recruitment

  2. Community monitoring

  3. Pay-for-performance

Inside One Health

Recruitment: Community selection versus “business as usual”

• Community selection: MAF facilitators hold village meetings to nominate and select the CAHW candidate

• Business as usual: CAHWs are recruited by state authorities in collaboration with Paramount Chiefs

• Minimal ability requirements (tests)

• Key design element: double listing

  • Obtain nominees from both Paramount Chief and community
  • Allows us to have the counter-factual

Recruitment and Incentives: Treatments

Community Monitoring

• CAHW and community develop joint action plans

• Interface meetings between ministry staff and community:

  • Two times: after 6 months and 1.5 years
  • CAHW performance revealed (relative and overall)
  • Allows community members to reward or impose social sanctions

Recruitment and Incentives: Treatments

Pay-for-Performance

• CAHWs are paid based on their rank (across three groups)
• Two times: after 6 months and 1.5 years

Note: rank ensure revenue neutrality but introduces small risk of non-independence / collusion

Recruitment and Incentives: Treatment Implementation

• Community monitoring and pay-for-performance implemented as designed

• The recruitment treatment not so much

  • Only in 100 villages do we have true double-listing (in many other cases either same person listed twice or no one listed by one party)

  • Implication: smaller sample and non-experimental interactions

Realized distribution of cases

• Small amount of attrition where candidates not identified (pre-randomization)

Recruitment and Incentives: Manipulation Check 1

• Does community selection impact CAHW type?
• Only modest differences from CAHW survey

CAHWs baseline survey

Recruitment and Incentives: Manipulation Check

• Does community selection impact CAHW type?
• More differences from HH survey

household baseline survey

Recruitment and Incentives: Manipulation Check

• Community monitoring and sanctioning

household baseline survey

Recruitment and Incentives: Manipulation Check

Recruitment and Incentives: Manipulation Check

Primary Outcome: Performance

Standard deviation of index in T2, T3 control condition: 0.19

Main design based results

Alternative specifcations

• Restricting T1 analysis to 100 pure randomizations
• T2 x T3 analyses without T1 and with randomization strata fixed effects

Subcomponents

Summary

• Evidence for both community monitoring and pay for performance
• No evidence for selection
• Noisy evidence for interactions: signs in wrong direction in two cases

Some of these results challenge the model more than others:

• No effect for T1 can be consistent with the model
• But we really do we expect from H 4: that more productive workers are more responsive to extrinsic incentives (T3=1).

Model based estimates of treatment effects

Posterior distributions on causal effects: close but note forced direction on interactions

Experiments: T1

Where and when might we expect stronger selection effects?

When might selection matter?

Experiments: T2

The Björkman and Svensson (2009) effect.

When might community sanctions work?

Experiments: T3

The payment effect.

When is payment most effective?

Discussion & Conclusion

Program

• Successful intervention: No increase in human health, but increase in good animal and human health practices, and animal stock

Institutions

• Evidence for two distinct institutional strategies
  • Consider now “invisible” costs
• No evidence here that selection changes outcomes
  • But plausibly case specific issues

Discussion & Conclusion

Explanation

• Highlights power of combining model and data for understanding lessons
• But also risks
• Perhaps set rival models in competition?

One Health Program

Created new positions in villages

One Health Program

Sustained engagement

Quality of Reporting

• Surveillance performance driven by events, effort and ability

• Records contain both Type 1 and Type 2 errors

• Both matter for containing pandemics

  • triggering responses to false events
  • failing to report events

Quality of Reporting

We measure the quality of CAHW reporting

• Practical: Central location to assess CAHW

  • Animal exam: CAHWs observe and diagnose sick animals, and record
  • Form review: Assess 5 (random) recently submitted forms

• Village visit: MAF representatives visit the CAHW’s village

  • Sick animal observation: Verify if sick animals had been reported
  • Form follow-up: Verify reported sick animals

Results: Quality of Reporting

Also: there is strong relationship between surveillance quality and performance

Quality

Results: Animal Husbandry Practices

- Data on disease control, response, knowledge, preventative practices